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[57] ABSTRACT 

The purpose of the present invention is to provide a hand- 
writing information detecting method and apparatus for the 
same, in which a user can choose any handwriting tool such 
as a pen or a pencil, and when the user writes with the tool, 
the handwriting information can be detected. The handwrit- 
ing information detecting method of the present invention 
comprises steps of detecting a motion of a writing tool, held 
by a writing tool holder comprising at least one acceleration 
sensor, based on at least one acceleration signal output from 
the at least one acceleration sensor when the writing tool is 
grasped by fingers; and recognizing handwriting informa- 
tion of a character or a figure written by the writing tool 
according to the detected motion thereof, and outputting the 
information. Preferably, the step of detecting a morion of a 
writing tool further comprises steps of detecting pressure 
acting on the tip of a finger which grasps the writing tool; 
and judging the motion of the writing tool according to the 
detected pressure. The handwriting information detecting 
apparatus of the present invention comprises a writing tool 
holder for detachabiy holding a writing tool; at least one 
acceleration sensor, provided at the writing tool holder, for 
detecting acceleration of the writing tool which is grasped 
by fingers; and a handwriting information recognition circuit 
for recognizing information of handwriting performed by 
the writing tool based on the output of the at least one 
acceleration sensor. 

37 Claims, 18 Drawing Sheets 
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HANDWRl'l'IiNG INFORMATION axial directions of the acceleration sensors can be deter- 

DETECTING METHOD AND APPARATUS mined when the user grasps the pen. without making the 

DETACHABLE HOLDING WRITING TOOL cross section of the pen a form which is difficult to grasp, as 

with the inverse triangle form. 

BACKGROUND OF THE INVENTION 5 Other objects of the present invention will be apparent by 

1. Held of the Invention me following description and the appended figures. 

The present Invention relates to a haixrwriting informa- Therefore, the present inveinion provides a r^vriting 

uon oVScctmgrnethoda^ Mormation, detecting n^^^r^steps effecting 

intthoaCiTider to input fcmdwriting information such as in a motion of a writing took held by a writing tool holder 

characters, figures, drawings, and so on produced by writing 10 comprising at least one accelcra^sens^. based on at least 

tools into a computer and the like. one acceleration signal output from the at least one accd- 

HArt erarion sensor when the writing tool is grasped by fingers: 

2. Background Art and recognizing handwriting information of a character or a 
Up to now. keyboards, mouses, and tablets have been used figure written by the writing tool according to the detected 

as devices for inputting character or figure information. In is mot ] on thereof, and outputting the informatioo. 

addition, as better portable inforniation-in^t o^ces. many Preferably, the step of detecting a motion of a writing tool 

apparatus have been proposed, in wtuchthe locus of a furthcr comprises steps of: detecting pressure acting on the 

character or a figure written by a pen grasped by a user is rf a ft™,^^ the writmg toca;andjuo^uiglhe 

detected by means of twice integrating output signakcf ^nof*^^ 

acceleration sensors which are built into the pen. and the 20 _ . . . . . . . u . . 

detected inform*** can be input into any computer (see ^ * ^""^Tf^' 

Japanese Patent Application ft* Publication Ud f^^nT^^^^^T 9 ^^ 

£l565U. He, 4-19^TL 4-256009. Ilei 4-282717 and for detachably hri^g a wnfcng too^aUcas, ooe accder*. 

^ Uon sensor, provided at the writing tool holder, for detecting 
soon ^ „ acceleration of the writing tool which is grasped by fingers: 
In such pens with built-in acceleration sensors, a pressure 25 ^ a hawfwriting information recognition circuit for rec- 
sensitive sensor may be attached at the axis of the pen for ogni ^ ng ^^^2^ c f handwriting performed by the writ- 
detecting up and down motions of the pen relative to the . tQol ^ mc QUtput ^ ^ at lea ^ acceleration 
paper or the surface of the desk (see Japanese Patent sensor. 

Application First Publication Na Hei ^ 1565 19). Preferably. X- and Y-axis acceleration sensors, the axes 

Usually, humans use different hand-writing tools suchas 30 correspooding to those of the two dimensional rectangular 

mechanical pencils, fountain pens, ball-point peas, marker coordinate assigned to a writing surface for the writing toot 

pens, water-paint pens, pencils, writing brushes, and paint- m provkkd as the at least one acceleration sensor, and the 

brushes according to their tastes or writing srnoothness. writmg ^ holder is grasped with the writing tool by the 

However, in the case of a pen with a built-in acceleration finger and has a grasping part for orienting the direction of 

sensor, there occurs a problem that the user is forced to use the writing tool holder so as to align the axial directions of 

a fixed and specific pen. the X- and Y-axis acceleration sensors in predetermined 

On the other hand, another kind of pen with built-in directions, 

acceleration sensors has also been proposed. In this case, a According to the present invention, any writing tool can 

ball-point cartridge is provided at the end of the pen. and ^ be ^4 the writing toc>ir*^ 

when a user writes on paper by using the pen, acceleration OD£ accda ^ on sensor thus, there is an advantage that each 

signals are processed to input the locus of the pen to a uscr caD the writing tool which he likes, 

computer (see Japanese Patent Application First Publication Aaditiwialry. because of the grasping part provided in the 

No. Hei 4-282717). However, in all the above cases, an apparatus, mc direction of the apparatus when the user 

exclusive pen has been needed and it has bceo impossible to ^ grasps the writing tool and the holder is oriented in a fixed 

change the pen. direction; thus, it is possible to ensure that each of at least 

On the other hand, another art has also been proposed, in one axial direction of the at least one acceleration sensor 

which the cross section of a pen is made to be an inverse rxovided at the writing tool holder is aligned in a predeter- 

triaogle in order that axial directions of the acceleration mined direction. Therefore, the user does not need to take 

sensors are easily determined when the user grasps the pen ^ cmc Q f the axial direction of the acceleration sensor, by 

(see Japanese Patent Application First Publication No. Sho which the level of comfort in using such apparatus is 

62-140 130). However, a problem with such a pen is mat it unproved. 

is hard to grasp in comparison with usual pens. following devices may be considered as fields of 

SUMMARY OF THE INVENTION application for the present invention: input devices for 

53 portable PDAs (Personal Digital Assistant); input devices 

Therefore, one object of the present invention is to for interactive pagers; voucher-writing devices, used in 

provide a handwriting information de terr i ng method and restaurants or express home-deliveries, for transmitting 

apparatus for the same, in which a user can choose any characters (including numerical characters) to the register 

handwriting tool such as a pen or a pencil, and when the user ^tfa the handwriting remaining on paper; and devices for 

writes with the too), the handwriting information can be ^ character-recognition of the contents of clinical records in a 

detected. hospital while the records are written on paper using a pen. 

Another object of the present invention is to provide a and for inputting the contents into a computer. 

handwriting information detecting method and apparatus for ™ ™^ ™ .«^ T ^o 

tte same!io which the downand up motions oftibe writing BRIEF EXPLANATION OF THE DRAWINGS 

tool can be correctly sensed. 65 FIG. 1 is a perspective view of the handwriting informa- 

Another object of the present invention is to provide a tion detecting apparatus of the first embodiment according to 

handwriting informatioQ detecting apparatus, in which the the present invention holding a pencil. 
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FIG. 2 is a perspective view of the detecting apparatus of 
FIG. 1 holding a pencil and being grasped by a user. 

FIG. 3 is a partially cut-away perspective view showing 
the internal structure of the detecting apparatus of FIG. 1. 

FIG. 4 is a flow chart explaining the input process for the 
writing points of the character recognition method in the first 
embodiment of the present invention. 

FIG. 5 is a flow chart explaining the recognition process 
of the character recognition method. 

FIG. 6A shows the real shape of a Japanese hiragana T* 
as an example of a handwritten character. FIG. 6B shows 
each writing point FIG. 6C shows interpotation of the 
writing points* FIG. 6D shows the division of each stroke, 
and FIG. 6E is for the purpose of explaining the determi- 
nation of matrix dements. 

FIG. 7 is a perspective view of the handwriting informa- 
tion detecting apparatus as a variation of the first embodi- 
ment 

FIG. 8 is a perspective view of the handwriting informa- 
tion detecting apparatus of the second embodiment of the 
present invention holding a pendL 

FIG. 9 is a perspective view of the detecting apparatus of 
FIG. 8 holding a pencil and being grasped by a user. 

FIG. It is a block diagram showing the handwriting 
recognition and display circuit of the detecting apparatus of 
FIG. 8. 

FIG. 11 is for the purpose of explaining a positional 
relationship between a pressure sensitive sensor fixed on an 
elastic beam which is projected from the body of the 
apparatus of FIG- 8. the index finger of a user, and a penciL 

FIG. 12 shows an arrangement for the experiment used to 
examine condition of the detection of the detecting appara- 
tus of FIG. 8. 

FIG. 13 shows the result of the above experiment 

FIG. 14 shows a variation of the second embodiment and 
explains a positional relationship between a strain gauge 
fixed on an elastic beam, the index finger of a user, and a 
pendL 

FIG. 15 shows another variation of the second 
embodiment, and explains a positional relationship between 
a pair of a light-emitting element and a light-receiving 
element fixed on an elastic beam, the index finger of a user, 
and a peadL 

FIG. 16 is a perspective view of the handwriting infor- 
mation detecting apparatus of the third embodiment of the 
present invention holding a penciL 

FIG. 17 is a perspective view of the drterting apparatus 
of FIG. 16 holding a pencil and being grasped by a user. 

FIG. 18 is a partially cut-away perspective view showing 
the internal structure of the detecting apparatus of FIG. 16. 

FIG. 19 is for the purpose of explaining starting and end 
points of the motion of one stroke during writing a character 
according to the third embodiment 

FIGS. 28A and 20B are for the purpose of explaining 
monitor images of the neighborhood of a starting point 

FIG. 21 is for the purpose of explaining a monitor image 
of the neighborhood of an end point 

FIG. 22 is a flow chart explaining the process for detect- 
ing motions of a pen by using the monitor images. 

FIG. 23 is a perspective view of the handwriting infor- 
mation detecting apparatus as a variation of the third 
embodiment 

DETAILED DESCRIPTION OF PREFERRED 
EMBODIMENTS 

Hereinafter, examples of the handwriting information 
detecting apparatus according to the present invention will 
be explained with reference to the figures. 
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First Embodiment 

The handwriting information detecting apparatus as an 
embodiment of the present apparat us will be explained 
5 below. 

Example of the Apparatus 

FIG. 1 is a perspective view of the apparatus holding a 
pencil. FIG. 2 is a perspective view of the apparatus being 
to grasped by fingers, and FIG. 3 is a partially cut-away 
perspective view showing the internal structure of the appa- 
ratus. 

In the figures, reference symbol A indicates the handwrit- 
ing information detecting apparatus of the present 

15 embodiment, reference numeral 1 indicates the body of the 
apparatus functioning as a writing tool holder, reference 
symbol a indicates a pencil as a suitable example of a 
writing toot reference symbol p indicates a hand and 
reference symbols pa and ($b indicate the index finger and 

20 the thumb of the hand, respectively. Reference numeral 2 
indicates a plate, reference numerals 3 and 4 indicate sup- 
porting pieces, reference numeral 5 indicates a binding pin, 
reference numeral 6 indicates a coil spring, reference 
numeral 7 indicates a writing tool clamping part reference 

23 numeral 8 indicates an L-step part reference numeral 9 
indicates an inclined part reference numerals 18 and 11 
indicate ends of the Instep part 8 and the inclined part 9 
respectively, reference numeral 12 indicates a handwriting 
information recognition circuit reference numerals 13. 14. 

30 and 15 indicate X-. Y-. and Z-axis acceleration sensors 
respectively, the axes corresponding to those of the three 
dimensional rectangular coordinates, and reference numeral 
28 indicates a power source. 

35 The handwriting information recognition circuit 12 
includes an amplifier 16 for amplifying and outputting each 
signal output from each of the X-. Y-. and Z-axis accelera- 
tion sensors, a two-times integrator 17 for twice integrating 
each amplified signal output from each of X- and Y-axis 
acceleration sensors and outputting the integrated signals 
and the amplified signal of the Z-axis acceleration sensor, a 
handwriting information recognizer 18 for recognizing 
handwriting information by detecting the motions of the 
writing tool based on the outputs from the two-times inte- 
grator 17. and a signal transmitter 19 for outputting the 

43 handwriting information which was recognized by the hand- 
writing information recognizer 18 to the outside of the 
apparatus. 

In the apparatus A. supporting pieces 3 and 4 are com- 
5q bined by binding pin 5 and elasttcally urged by coil spring 

6 so as to enable ends 1# and 11 to elasticalry open and dose 
in a way similar to a dothes-pin. That is. as shown in FIG. 
1. pencil a is damped and held by writing tool damping part 

7 like the case of a dothes-pin. 

55 As a writing tool to be damped by writing tool clamping 
part 7. a ball-point pen. a ntfchanical penciL a fountain pen. 
a writing brush, a marker pen. a water-paint pen. a 
paintbrush, and the like are also possible to choose in place 
of pencil cl 

60 Z-axis acceleration sensor 15 is arranged in a manner such 
that the direction of its axis is approximately made paralld 
to the line of the end 18 of the L-step part 8. that is. made 
approximately paralld to the direction of up and down 
motions of the writing tool grasped by the user. On the other 

€5 hand. X- and Y-axis acceleration sensors 13 and 14 are 
arranged in a manner such that the direction of each axis is 
perpendicular to the axial direction of the Z-axis accdera- 
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tioo sensor 15 and that the axial directions of the X- and 
Y-axis acceleration sensors arc also papesdiculaT to each 
other. These three sensors are built in the body 1. 

When a user uses the writing tool (here, pencil a) held by 
apparatus A. body 1 is gasped by the user in a manner of 
putting the body between his ringers* by applying his thumb 
pb to inclined part 9. while applying his index finger pa to 
L-step part ft* as shown in FIG. 2. According to this, the 
location (direction) of apparatus A while being grasped is 
always approximately fixed; thus, the axial directions of X-. 
Y-. and Z-axis acceleration sensors which are built in the 
body 1 are aligned in predetermined directions approxi- 
mately. 

Power source 2t may have ordinary dry cells, or lithium. 
Ni — Cd. or air cells. Solar cells, which are attached on the 
body 1, may also be used. 

Method Example 

Hereinbetow. a method example for drtecring handwrit- 
ing information of the present invention, which is applied to 
the present apparatus, will be explained. 

When me user writes a character and so on with pencil a 
which is held by apparatus A. the body which is combined 
with the pencil a moves together with the pencil 

First, the up and down motions of pencil a are de t e ct e d by 
using signal S z output from Z-axis acceleration sensor 15. 
Here, the following definitions arc provided for explanation. 
The down morion of the writing tool indicates a motion of 
lowering the writing tool so as its point touches the paper, 
while the up motion of the writing tool indicates a motion of 
raising the writing tool so as to remove its point from the 
paper. Additionally, a "stroke" indicates a morion which 
occurs between a down motion and me following up motion 
of the writing tool. 

In this example, two thresholds for the down and up 
motions are provided for the level of signal and if the 
signal level exceeds die threshold for the up motion, it is 
judged that the up motion has occurred, while if the signal 
level exceeds the threshold for the down motion, it is judged 
mat the down motion has occurred Here, according to 
which directions of the up and down motions the positive 
direction of Znaxis acceleration sensor 15 corresponds to. 
one of the thresholds for the down and up motions is 
positive, and the other is negative. In each motion, the 
absolute value of the signal varies in a manner such mat the 
value exceeds the corresponding threshold. In addition, the 
absolute values of both thresholds may be different or may 
be the same. According to this rnethod.it is rx»ssiUe to detect 
each remarkable point of the motions of the writing tool, 
such as a starting point of writing a character or a figure, a 
rest, a restarting point of the writing, the end point of writing 
a character or a figure, and the like. 

Next motions of the writing tool on the paper (Le.. on the 
surface for writing) are detected by using signals S*. S y 
output from X- and Y-axis acceleration sensors 13 and 14. If 
time variations of acceleration of X- and Y-axes indicated by 
these signals are twice integrated and the results (that is. 
displacement data for the X- and Y-dnxctions) are repre- 
sented in the X-Y coordinate plane, the character or figure 
which was written by the user can be restored. A concrete 
example of character recognition using such a method will 
be explained below in connection with a general pen. 
referring to the flow charts of FIGS. 4 and 5. Here. FIG. 4 
shows the input process for writing points (Le.. points on the 
stroke), and FIG. 5 shows the recognition process based on 
the inputted writing points. Each process is performed by 
handwriting information recognizer 18. 
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The input process commences along with the initial 
assignment of stroke number n=l (see step SP101). and the 
hand^mting input of the nth stroke of a character is con- 
ducted using the apparatus A (sec step SP1#2). This char- 
5 acter is designated as Cx. 

Next, it is judged whether or not the point of the pen is in 
contact with an object for writing (see step SP1*3). If judged 
"yes", the coordinates of the pen point are recorded for the 
first writing point of the sequence information of the nth 
10 stroke (sec step SP1+4). These coordinates are determined 
according to the displacement data output from two-times 
integrator 17. 

On the other hand. If judged *W in step SP1#3. non- 
contact time r with regard to the pen and me object for 
writing (that is, the time in which the pen does not touch 

13 with the object) is measured (see steps SP1*5-SP107). 
Then, it is judged whether or not the non-contact time r is 
smaller man a preset value R (see step SP108). and when the 
non-contact time r reaches or exceeds this preset value R. it 
is recognized that handwriting input of all strokes of the 

20 character Cx has been completed, and the recognition pro- 
cess from step SP2#1 comtnences. 

On the other hand, in the case when the pen point comes 
back into contact with the paper while the non-contact time 
r does not reach preset value R. it is recognized mat 

25 handwriting input of the (n+l)th stroke has rrtmmen ced. and 
input processing continues (sec steps SP16&. SP106. and 
SP109). 

Here, it is assumed that the user wrote a Japanese hiragana 
**r (here, represented in Romaji). whose real shape is shown 

30 in FIG. 6A. FIG. 6B shows writing points whose coordinates 
were recorded by the above input process. 

Id step SP201 in FIG. 5. the neighboring points of each 
respective stroke recorded above are connected using 
straight lines to interpolate each interval between the stroke 

33 points. This state is shown in FIG. 6C. 

Subsequently, such polygonal line obtained by such link- 
age for each stroke is equally divided into k(k^l) polygonal 
(or straight) lines, and numbers are assigned to each division 

^ point. Le~ L 2 k+1 (1 is the stroke origin; k+1 is die 

end of the stroke) according to the time sequence of the 
points (see step SP292X The result in a case of k=4 for FIG. 
6C is shown in FIG. 6D. 

Next, taking any two strokes n 1 and n? from all strokes in 

45 turn, a (k+l)N rowxk-N column (where N is the number of 
strokes) matrix P consisting of elements is calculated, 
wherein die drment of row {(k+lXn,— l>+i} and column 
{k( n^-l comprises an angle Q formed by means of a 
straight line joining division point i of stroke n x and division 

30 point j of stroke n^. and a straight line joining m* vision point 
i of stroke n L and division point j+1 of stroke n? (sec step 
SP2i3). 

In FIG. «X N=2. fc=4. n 4 =l. and n^=2; thus, a 10(row)x8 
(column) matrix P consisting of elements is calculated. 

55 wherein the element of the row i and column (4+j) comprises 
the angle Q formed by means of a straight line joining 
division point i (r=l~5) of stroke n A and division point j 
(£=1-4) of stroke n^. and a straight line joining division point 
i of stroke a t and division point j+1 of stroke n^ The angle 

60 Q shown in FIG. 6E comprises t=2 and j=3; thus the angle 
is determined as the element of row 2 and column 7 of 
matrix P. 

In handwriting information recognizer 18 of apparatus A. 
similar matrices corresponding to each character are stored. 
65 and they will be referred to as templates. 

The distance di (i=l. . . . . M) between the inputted 
handwritten character Cx and M templates Qi (i=l M) 
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corresponding to the respective characters possessing the 
same oumbcr of strokes is calculatfd as the sum of the 
squares of each element of a matrix which is obtained by 
"matrix P-matrix Qi (see step SP2M). Here, this sum of the 
square is calculated by means of either taking me sum of the 
squares of all dements of "matrix P-matrix Qi**, or taking 
the sum of the squares of a specific portion of the above 
matrix (eg., the portion of elements calnrtatrd from the 
straight line connecting a division point of stroke n and a 
division point of stroke n+1). 

The character Cz correspooding to the template Qz with 
the smallest distance dz is then selected as the recognition 
result (see step SP2t5). If other characters for recognition 
exist the procedure returns to step SP101. and the above- 
described steps are repeated (see step SP2S6). 

It is also possible to recognize the handwriting informa- 
tion according to velocity data which are obtained by once 
integrating the acceleration data, instead of twice integrating 
those data. In this case, in step SP104 of me input procedure 
shown in FIG. 4. components v x and v, of the velocity v of 
the pen point moving along each stroke may be input as 
coordinate data instead of coordinates of the writing point. 
In this case, templates corresponding to the velocity data 
must be previously stared. 

B is also possible to recognize the handwriting informa- 
tion according to acceleration data themselves. In this case, 
suitable templates are previously stored, and components a x 
and a r of the acceleration a of the pen point moving along 
each stroke are input as coordinate data. 

Combination of two or more of these three kinds of 
methods is also possible for recognizing the handwriting 
information. 

The recognized character information may be transmitted 
out from signal transmitter 19 in the form of character codes 
via a wireless or infrared system and be received by any 
external and ordinary computer which has a receiver for the 
information. The signal transrnission from signal transmitter 
19 may be performed via a wired system. 

On the other hand, the output from two-times integrator 
17. that is. the result of the two-times integration, which is 
the locus of a character or figure itself, may be directly 
output from the signal transmitter 19 without recognizing 
the character or figure based on the above-mentioned writing 
points. 

Variation of the Apparatus 

FIG. 7 is a p er sp ec tive view of handwriting information 
detecting apparatus B as a variation of apparatus A in FIG. 
L In the figure, reference numeral 21 indicates a body of the 
apparatus, reference numeral 22 indicates an indentation, 
provided at an end of the body 2L for holding a writing tool, 
reference numeral 23 indicates screws for pressing and 
fixing the writing tool which is inserted in the indentation 22 
by their points, via tapped holes 24 with spot facing which 
arc provided outside body 2L 

Therefore, apparatus B detachahfy holds any writing tool 
by using indentation 22 of body 21 and screws 23. not by 
using elastic force of the coil spring as the case of apparatus 
A. 

In the body 21. the same acceleration sensors and the 
handwriting information recognition circuit as those of the 
apparatus A shown in FIG. 3 are internally provided and the 
procedures explained in the above "Method Example" are 
performed for recognizing the handwriting information. 

Second Embodiment 

In the method of the first embodiment there occurs a 
problem that if sufficient displacement or impact in the down 
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direction does net occur during the down motion, die motion 
cannot be properly detected. In addition, in the up motion of 
the writing tooL there occurs further smaller impact in 
comparison with the case of the down motion; thus, the 
5 detection is harder. The second cnnVxtiment for improving 
this point will be shown below. 

Example of the Apparatus 

FIG. 8 is a perspective view of the handwriting informa- 

10 tion detecting apparatus of this embodiment holding a 
pencil. FIG- 9 is a perspective view of the apparatus being 
grasped by fingers. FIG. 10 is a block diagram showing the 
handwriting recognition and display circuit of the a p para t us, 
and FIG. 11 is for the purpose of explaining a positional 

15 relationship between a pressure sensitive sensor fixed on a 
beam which is projected from the body of the apparatus, the 
index finger of a user, and the pencil. In these figures, parts 
whkh are identical to those shown in FIGS. 1-3 are given 
identical reference numbers and an explanation thereof will 

20 be omitted here. 

In the FIGS. 8-11. reference symbol C indicates the 
handwriting information drtcrting apparatus of the present 
embodiment, reference symbol pa* indiratrs the finger cush- 
ion of the tip of index finger pa. reference numeral 31 
indicates the body of the apparatus forming a curved step 
part 3Ut. the body functioning as a writing tool holder, and 
reference numeral 32 indicates an elastic beam, made of 
plastics and the like, which is projected from the curved step 

^ part 31a of the body 31. The finger cushion pa' of the index 
finger touches the end of the beam when the user grasps 
pencil cl Reference numeral 33 indicates a pressure sensi- 
tive sensor, fixed on the end of the elastic beam 32. for 
detecting the level of the pressure caused by finger cushion 

Reference numeral 34 indicates a character display, and a 
liquid crystal display is used in this embodiment. Reference 
numeral 36 indicates a writing tool holding part comprising 
a pair off rings 37 and 38 for inserting and boldmgth^ writing 

40 tool, a block nut 39 which is fixed between the lower ends 
of these rings, and a bolt 4$ for fixing the writing tooL In the 
case of apparatus C. pencil a is inserted into rings 37 and 38 
to a suitable position, and by fastening bolt 40. the pencil a 
is fixed to the apparatus C. 

43 Next, reference numeral 51 indicates a handwriting rec- 
ognition and display circuit which is btnlt in the body 31. 
Reference numeral 55 indicates an amplifier for amplifying 
and outputting each signal output from each of X- and Y-axis 
acceleration sensors, reference numeral 56 indicates a two- 

50 times integrator for twice integrating each amplified signal 
output from each of X- and Y-axis acceleration sensors and 
outputting the integrated signals, reference numeral 57 indi- 
cates a handwriting information recognizer for recognizing 
handwriting information by detecting motions of the writing 

55 tool based on the outputs from the two-times integrator 56. 
ref erence numeral 58 indicates a signal transmitter for 
outputting the handwriting information which was recog- 
nized by the handwriting information recognizer 57. and 
reference numeral 59 indicates a signal receiver which can 

60 receive signals transmitted from any external device. 

Reference numeral 61 indicates a CPU (central processing 
unit), connected to pressure sensitive sensor 33. for totally 
controlling the whole system including amplifier 55. two- 
times integrator 56. handwriting information recognizer 57. 

65 signal transmitter 58. and signal receiver 59. and reference 
numeral 62 indicates a power source, using solar cells 35 
which are mounted on the body, for supplying drive current 
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to tbe whole system. Power source 62 may have ordinary dry information. At this time, the recognized result of the 

cells, or lithium. Ni — Cd. or air ceils in the body. An external character written by the user is displayed on character 

power source may also be used via an electric cable. display (liquid crystal display) 34; thus, the user can confirm 

Whcnauu^t^wriu^^ %Sy to tte character iimut 

lower part of ^5^?°*^ ^ *^!!nM»«M* ^^na!^'!* rne^^ 

E^ 8 ^ 1 ^^ ^ ^writing tool holding part 3^ ^ craved via a wireless orinfrared system 

between his thumbpb and into finger pa. as show mHG. fro.nTcommiiiik^ partner may be received by signal 

9. Also* this case. the rx>si^ KC ^ va & ^ k ^ ^ ^ to read the message 

grasped is always approximately fixed; thus, the axial three- 0D character display 34. 

tions of X- and Y-axis acceleration sensors 13 and 14 which 10 

are built in the body 31 are aligned in pre-determined Experimental Example 

directions approximately. Now. an experiment to examine condition of the detection 

Additionally, at this time, the positional relationship of the apparatus and its results will be explained below. In 

between the index finger Ba of me user, elastic beam 32. ** esperiinent. as shown in FIG. 12. ball-point pen BP was 

wessurc sensitive senX^airf pencu a is stich as shown 15 beld by the handwriting information detecting apparatus C 

^r^r^T. ^r^^Ki^fw^F *L ««<~ fu and atestee was instructed to perform a writing action on a 

in FIG. 8; thus, finger cushion pa of the index finger pa *f 

touches with the pressure sensitive sensor 33. ^ r . „ \\ . c v . ^ 

^ The pen point of ball-pant pen 1% is made of metal; thus. 

Method Example while the pen point was in contact with the metal plate 65 

20 (that is. in the down state of the pen), voltage of battery 66 

Hereinbelow. a method example for detecting the hand- was detected as signal S*. On the other hasd. me output from 

writing information of tbe present invention, which is pressure sensitive sensor 33 of apparatus C was detected as 

applied to the present apparatus, will be explained. signal S A . 

When the user writes characters by pencil a on a writing FIG. 13 shows variations of bom signals obtained from 

surface like a paper on the desk, in the "down state" in which 25 one stroke performed by a testee. In the graph of FIG. 13. the 

the point of pencil a contacts with the paper, considerable horizontal axis indicates "time", while the vertical axis 

force acts on finger cushion pa* of index finger pa which indicates "voltage". Here, signal S A has a negative charac- 

contacts with pencil a. On the other hand, in the "up state** teristic 

in which the point of pencil a does not touch with the paper. According to FIG. 13. it is understood that a pen-down 

the force which acts on the finger cushion pa* is smaller than 30 motion was performed at about time t x . and a pen-up motion 

that of the down state. was performed at about time t r Here, the outputs from 

Accordingly, the up and down motions of pencil a to the pressure sensitive sensor 33 very much corresponds to the 

paper etc can be detected by processing actual peo-motioa which is represented by signal For 

pressure sensitive sensor 33 mmmted on elastic beam 32 other testers, similar results of satisfactory correspondence 

based on thresholds, as the case of the first embodiment 33 were obtained. 

That is to say. if the signal level exceeds a threshold for the Therefore, it is understood that for example, the follow- 
down motion, that is. if the pressure exceeds a level, it is ing judgment can be adopted: if signal S A . the output from 
judged that the down motion has occurred, while if the pressure sensitive sensor 33. goes below -0.5 V. the motion 
signal level goes below a threshold for the up motion, mat is "pen-down", while if signal S A exceeds -0.5 V.the motion 
is. if the pressure goes below a level, it is judged that the up 40 is "pen-up". 

^ !T ° CC !^ ACCQ, ^ 8 .^ > ^ h Fast Variation of the Apparatus 
detect each remarkable point of the motions of the writing 

tool, such as a starting point of writing a character or a FBG- 14 shows a variation of apparatus C in FIG. ft. and 

figure, the end point of writing a character or a figure, and explains a positional relationship between a strain gauge 

the like. Here, ifme output signal from trie pressure sensitive 45 fixed on the elastic beam, the index finger of a user, and the 

sensor 33 has a negative charactmstic. the jud^memmay be pencil. In the figure, reference numeral 73 indicates a strain 

conducted such that if the signal exceeds a tbreshold in the gauge fixed on a part near the root of elastic beam 32. That 

negative direction, the motion is judged as a down motion, & hi this variation, the strain gauge 73 is provided as a 

Ne*. motions oo the surface for writing are detected by sensor far judging the up and down n»tK*sof vcr^a. 

using signals S*. S, output from X- and Y-axis acceleration 50 <* I-es^ire sensitive sensor 33 in FIG. 8- Other 

sensors 13 and M. The recognition of handwriting informs- anatigements of tbe variation are Che same as those of 

tion based on these signals are performed by handwriting "pparatus C 

information recognizer 57 and any one of the following four As described above. Much more force acts on finger 

methods can be adopted, as the case of the first embodiment, cushion pa' of index finger pa which contacts with pencil a 

(1) Using tbe twice integrated data of accderatio. of X- 55 *° *"» ° f I*?? « ^ ^t^ 8 
aiKJY-Gffections during the up state of the pencil; thus, elastic beam 32 bends 

^ v „. . . ^ v and is strained. The quantity of the strain is measured by the 

(2) Using me once integrated daU of acceieratto. of X- ^ ^ n ^ ^ ^ ^ jn ^ p^m of pencil 

and Y-directions a does not touch with the writing surface, the strain on 

(3) Using the acceleration data of X- and Y-dxrectkms 60 clastic beam 32 becomes small. 

themsdvcs Therefore, motions of pencil a can be detected by per- 

(4) Using a cornhination of two or more of the above three fanning a similar threshold process to the above-explained 
kinds of methods examples for the output signal from strain gauge 73. 

The recognized character information may be transmitted 

oijt&om signal traiisrnitter 59 m 65 Second Variation of the Apparatus 

via a wireless or infrared system and be received by any FIG. 15 shows another variation of the apparatus C and 

external and ordinary computer which has a receiver for the explains a positional relationship between a pair of a light- 
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emitting dement and a light-receiving p*m*>nt fixed on the grasped by fingers* and FKL 18 is a partially cut-away 

elastic beam, the index finger of a user, and the penciL In the perspective view showing the internal structure of the appa- 

figure. reference numerals 74 aod 75 indicate a LED (light ratus. Id these figures, parts which are identical to those 

emitting diode) and a phototranastor respectively, as suit- shown in FIGS. 1-3 and FIGS- 8~1# are given identical 

able examples of a pair of a light-emitting element and a 5 reference numbers and an explanation (hereof will be omit- 

light-receiving element They are fixed on the end part of nerc * 

elastic beam 32. That is. in this variation. LED 74 and hi the FIGS. 16-18. reference symbol D indicates the 

phototransistor 75 are provided instead of pressure sensitive hamrwrmng information dct«^ng apparatus of ^ P^ff* 

sensor 33. and other arrangements of the variation are the «nbo<fin>ent, reference numeral 31"**^^^ <* fec 

same as those of apparatus C ic " * ^ ,*? ^ 

. v7 ^. ^ . ^ _ ^ WTOfc numeral Sla indicates a CCD (charge coupled device) 

In this ca^i^ m^redhght wfoch * oraduted by LED cont ^ ^ ^ fQrmed as a part of the body 81. Refer- 

74 is scattered inside the finger, aod a p&rt of the scattered ^ numCfal 99 indicates a CCD monitor camera as a 

^ ^ d f w f d ^pbototonsistor75. The variato of the cxamp|c rf a pboio&zpbic ^tor means and is 

level of the detected light corresponds to the phctodectrtt housed in the CCD container 81a. Reference numeral 95 

pulse wave. The ph^ectnc pulse wave is exrJained such » indicates a CPU. coonected to CCD monitor camera 98. for 

that the vanauon of blood pressure Jfcccdmg to the beat of u ^ TOntrolling ^ including amplifier 55. 

&e heart is detected as a P^oo of the ^vessel. Here^ twcMimes integrator 56. handwriting information recognizer 

^tdfm^pcssuic inside and outside the Wood vessel ^ ^ t^^^ 58. and reference numeral 96 

of the artery greases, the compliance of ^ artery a $omcc fQg ^vc current to the 

uk«^; thus, the ann^ 20 system. The apparatus D has the same mechanism as 

nic pulse wave also ii»eases(Hiraiw ^ ^ apparatu$ a. m shown in Fia 16. pencil a is 

Stress Change by Means of Noninvasive Mjagurement of cbaB ^ ^ ^ ^ ^ da^g part 7. Here, a 

Arterial ftessure and Mecha^ suftab|c ho|dm ^ ^ ^ for m0 m^ormg iniages of 

nigs of 3rd Symposium on Human Interface. PP <H-66. <nc neighborhood of the point of pencfl a by CCD i*Hiitor 

Society of Instrument and Control Engineers. 1987). 23 camera9& 

Therefore, when the point of pencil a contacts with the xhc manner of using pencil a by a user is similar to that 

surface for writing, LED 74 and phototransistor 75 presses of apparatus A. and the direction of apparatus D when the 

into the finger cushion ^ of the index finger pa of the user. ^ grasps pencii a & t&so always approximately fixed; 

and the pressure which is added to the finger cushion Pa* mus ^ directions of X- and Y-axis acceleration 

increases; thus, the differential pressure inside and outside sensors 13 and 14 are aligned in pre^oetermiiied directions 

the Mood vessel in the finger cushion Pa' decreases. That is. approximately, 
the amplitude of the signal of the photcclectric pulse wave. 

which has been photCHdectrically converted by phototrans- Method Example 

istor 75 from near infrared light which had passed through Hereinbelow. a method example for <tcftrtff»g the hand- 

the skin, increases. On the other hand* when the point of 35 writing information of the present invention, which is 

pencil a does not contact with the surface for writing, the applied to the above apparatus, will be explained. FIG. 19 is 

pressure added to the finger cushion pa' of the index finger for the purpose of explaining the starting and end points of 

pa decreases; mus. the differential pressure inside and out- the motion of one stroke during writing a character. FIGS, 

side the bloodvessel in the finger cushion pa* increases. That 2#A and 20B are for the purpose of explaining monitor 

is. the amplitude of the signal of the photoelectric pulse 40 images of the area around a starting point FIG. 21 is for the 

wave decreases. purpose of explaining a monitor image of the area around an 

Accordingty. also in this case, by performing a similar end point and FIG. 22 is a flow chart explaining the process 

threshold process to the above-explained examples for the for detecting motions of a pen by using the monitor images, 

amplitude of the photoelectric pulse wave signaL the judg- 45 Hereinafter, it is assumed that a right-handed user writes 

ment can be performed such that if the amplitude exceeds a a character or a figure by using an arbitrary pen held by the 

threshold, the down motion of pencil a has occurred, while present apparatus, and judgments for the down and up 

if the ajmpifrmfr goes below a threshold, the up motion of motions, and for the starting and end points will be 

pencil a has occurred. explained. In the consecutive image processing. CPU 95 

As explained above, in this emr>odiment the variation of x takes charge of the timing control between the parts of the 

the pressure acting on the finger cushion of the index finger system. 

of the user during the up or down motion of the writing tool As shown in FIG. 19. One stroke ST during writing a 

is detected by an elastic beam and various kinds of sensors; character has a writing starting point S and a writing end 

thus, there is an advantage mat the up and down motions of point E. CCD monitor camera 98 takes pictures of the view 

any writing tool can be certainly and easily detected. 55 of neighborhood of pen point al. as shown in FIG. 2#A.The 

dotted circle near the center of the circular monitor image 

Third Embodiment MD will be called as center circle SC. 

Similar to the above second embodiment an example of Hereinbelow. the process will be explained according to 

the hamrwritiiig information detecting apparatus and method the flow chart of FIG. 22. First CCD camera 98 takes up an 

in which the up and down motions of the writing tool arc «j image of time t as shown in FIG. 2tA (see step SP1). Here, 

detected without providing an acceleration sensor fox the a inicro-time is assigned as At (for example. At=33 [msec]). 

Z-direction will be proposed below. When the micro-time At passes from time t a similar 

image is taken up again (see step SP2). The difference 

Example of the Apparatus between the above two images is then calculated (see step 

FIG. It is a perspective view of the handwriting infor- 63 SP3). 

mation ^r^iTg apparatus of this emhodimrnt holding a Next for both of the images of times t and t+At it is 

penciL FIG. 17 is a perspective view of the apparatus being judged whether or not the inside of center circle SC except 
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the peo point is Wane (see step SP4). If judged **yes*\ it is 
recognized that the pen exists in the air (Le.. in the pen-up 
state) at time t (see step SP5). time t is then renewed as 
**t=t+AT (see step SFf), and the process returns to step SP1. 
On the other hand, if judged "no", it is further judged 5 
whether or not images of t and t+At differ from each other 
in connection with the black line in center cirde SC (see step 
SP7). 

If judged "yes" in step SF7, it is recognized that the pen 
performs a stroke (Le.. in the pen-down state) at time t (see 10 
step SP8). time t is then renewed as "t=t+AT (see step SP9), 
aad the process returns to step SP1. On the other hand, if 
judged "no** in step SP7. it is further judged whether or not 
the inside of center circle SC except the pen point is Mane 
with regard to the image of time t (see step SP10). 15 

If judged "yes** in step SP10. it is recognized that time t 
is a starting point of writing of the pen (Le., the moment of 
the pen-down motion) (see step SP11). time t is then 
renewed as "t=t+AT (see step SP12). and the process returns 
to step SP1. On the other hand, if judged "no** in step SP10. 
it Is recognized that time t is an end point of a stroke (see 
stepSP13). 

Next, it is judged whether any writing does not occur in 
a further tinaT or inc«e (for example. T=l [msec]) (see step ^ 
SP14). This judgment is performed in a manner similar to 
the above process for judging whether or not the inside of 
center circle SC except the pen point is blanc with regard to 
the image of time t If judged that no writing has occurred, 
it is recognized that the writing of the character has finished ^ 
(see step SP1*), and the process ends. On the other hand, if 
judged that some writing has occurred, time t then renewed 
as M t=ttAf* (see step SP15) and it is recognized that the pen 
is in the course of proceeding to a next stroke, and the 
process returns to step SP1. 33 

For example, if an image as shown in FIG. 20A is 
obtained at time t and another image as shown in FIG. 20B 
is obtained at time t-fAt. time t is recognized as the starting 
point of a stroke. On the other hand, if the same images as 
shown in FK3. 21 are obtained at times t and t+At. time t is 40 
recognized as the end point of a stroke. 

Now the explanations for the judgments of the starting 
and end points of the stroke and the judgments of the 
pen-down and pen-up are concluded, 

Regarding the method for detecting the motions of pencil 45 
a based on acceleration signals S* and S, output from X- 
and Y-axis acceleration sensors 13 and 14 so as to recog- 
nizing the handwriting information, same method as that of 
embodiment 1 or 2 can be adopted. 

Additionally, also in the apparatus D. a display made of 50 
LCD and so on may be provided on the body to visually 
feedback the recognized characters to the user. 

It is also possible to provide a signal receiver as embodi- 
ment 2 so as to receive signals transmitted from an external J5 
device like a pager and to display the information corre- 
sponding to the signals on the above display. 

Another method of using an image pick-up tube monitor 
and the Uke is possible for taking monitor image MD around 
the pen point oil. besides the method of using CCD as the ^ 
present embodiment. 

Variation of the Apparatus 

FIG. 23 is a perspective view of the handwriting infor- 
mation drtrcting apparatus E as a variation of apparatus D 69 
in FIG. 16. In the figpre. reference numeral 101 indicates a 
body of the apparatus, and reference numeral 101a indicates 



a CCD container which is formed as apart of the body 10L 
Reference numeral 102 indicates an indentation, provided at 
an end of the body 101. for holding a writing tool, reference 
numeral 103 indicates screws for pressing and fixing the 
writing tool which is inserted in the indentation 102 by their 
points, via tapped holes 104 provided outside body 10L 

Therefore, apparatus E detachabfy holds any writing tool 
by using indentation 102 of body 101 and screws 103. not 
by using elastic force of the coil spring as the case of 
apparatus D. Other parts of apparatus E are the same as those 
of apparatus D which were explained above. 

As explained above, in this third embodiment the CCD 
monitor camera contained in the apparatus monitors and 
takes pictures of the neighborhood of the pen point of the 
writing tool and the handwriting information is detected 
based on the monitored images; thus, mere is an advantage 
that the up and down motions of any writing tool can be 
certainly recognized. 

What is claimed is: 

1. A handwriting information detecting method compris- 
ing the steps of: 

dctachabry holding a user selectable writing tool in a 
writing tool holder comprising at least ooe acceleration 
sensor; 

detecting a motion of the writing tooL held by the writing 
tool holder, based on at least one acceleration signal 
output from the at least one acceleration sensor when 
the writing tool is grasped by fingers; and 

recognizing handwriting information of a character or a 
figure written by the writing tool according to the 
detected motion thereof, and outputting the informa- 
tion. 

2. A handwriting inf ormatkn detecting method as claimed 
in claim t> wherein: 

X- and Y-axis acceleration sensors, the axes correspond- 
ing to those of the two dimensional rectangular coor- 
dinates assigned to a writing surface for the writing 
tooL are used as the at least one acceleration sensor; 
and 

the writing tool holder is oriented to a predetermined 
direction when the writing tool and the holder are 
grasped by the fingers, whereby the axial directions of 
the X- and Y-axis acceleration sensors are aligned in 
predetermined directions. 

3. A handwriting inf conation detectmg method as claimed 
in daim 2, wherein a Z-axis acceleration sensor whose axial 
direction corresponds to the direction of the axis of the 
writing tool is further used as the at least one acceleration 
sensor 

4. A handwriting information detecting method as claimed 
in claim 3. wherein the axial direction of the Z-axis accel- 
eration sensor is perpendicular to each of the axial directions 
of the X- and Y-axis acceleration sensors. 

5. A handwriting information detecting method as claimed 
in claim 3. wherein: 

the step of detecting a motion of a writing tool is per- 
formed in a manner such that in case the acceleration 
signal output from the Z-axis acceleration sensor is a 
positive signal in an up motion of the writing tool, if the 
positive acceleration signal exceeds a predetermined 
positive threshold, it is judged that the writing tool is in 
an up-state, while if the negative acceleration signal 
exceeds a predetermined negative threshold, it is 
judged that the writing tool is in a down-state; 

while in case the acceleration signal output from the 
Z-axis acceleration sensor is a negative signal in an up 
motion of the writing tool, if the negative acc el era t ion 



